Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [MongoDB] Support multiple type field #540

Merged
merged 4 commits into from
Nov 23, 2023

Conversation

mrtc0
Copy link
Contributor

@mrtc0 mrtc0 commented Nov 22, 2023

Issue Context / why we need it

MongoDB is schema-less, so field may have different types. When dealing with such databases, the $sample stage may produce different results with each execution.

For example:

$ cat init/init.js
db.createCollection('tblstest');
db.tblstest.insertMany(
  [
    { name: 'test1', age: 1 },
    { name: 'test2', age: 2 },
    { name: 'test3', age: 3 },
    { name: 'test4', age: 4 },
    { name: 'test5', age: null },
    { name: 'test6', age: null },
    { name: 'test7', age: null },
    { name: 'test8', age: null },
    { name: 'test9', age: 9 },
    { name: 'test10', age: 10 },
  ]
)

$ docker run --rm -it -p 27017:27017 -v $PWD/init:/docker-entrypoint-initdb.d mongo:7.0.3

$ cat .tbls.yml
dsn: mongodb://localhost:27017/test?sampleSize=10
docPath: ./docs/schemas/mongodb
er:
  skip: true

# Repeat multiple times...
$ tbls doc --rm-dist
...

$ git diff
diff --git a/docs/schemas/mongodb/test.tblstest.md b/docs/schemas/mongodb/test.tblstest.md
index d5987f7..881552b 100644
--- a/docs/schemas/mongodb/test.tblstest.md
+++ b/docs/schemas/mongodb/test.tblstest.md
@@ -9,7 +9,7 @@ Count of documents is 10
 | Name | Type | Default | Nullable | Occurrences | Percents | Children | Parents | Comment |
 | ---- | ---- | ------- | -------- | ----------- | -------- | -------- | ------- | ------- |
 | _id | objectId |  | false | 5 | 100.0 |  |  |  |
-| age | int32 |  | false | 5 | 100.0 |  |  |  |
+| age | <nil> |  | false | 5 | 100.0 |  |  |  |
 | name | string |  | false | 5 | 100.0 |  |  |  |

I think it's bothersome to have differences with each execution, so it would be preferable to either suppress this or output all types of the aggregated documents.

What this PR does

From the documents selected in the $sample stage, all types of the fields will be enumerated with commas separated.
This can be enabled with the multipleFieldType query.

For example:

$ cat .tbls.yml
dsn: mongodb://localhost:27019/test?sampleSize=10&multipleFieldType=true
docPath: ./docs/schemas/mongodb
er:
  skip: true

$ tbls doc --rm-dist
$ git diff
diff --git a/docs/schemas/mongodb/test.tblstest.md b/docs/schemas/mongodb/test.tblstest.md
index d5987f7..9e30dda 100644
--- a/docs/schemas/mongodb/test.tblstest.md
+++ b/docs/schemas/mongodb/test.tblstest.md
@@ -8,9 +8,9 @@ Count of documents is 10

 | Name | Type | Default | Nullable | Occurrences | Percents | Children | Parents | Comment |
 | ---- | ---- | ------- | -------- | ----------- | -------- | -------- | ------- | ------- |
-| _id | objectId |  | false | 5 | 100.0 |  |  |  |
-| age | int32 |  | false | 5 | 100.0 |  |  |  |
-| name | string |  | false | 5 | 100.0 |  |  |  |
+| _id | objectId |  | false | 10 | 100.0 |  |  |  |
+| age | <nil>,int32 |  | false | 10 | 100.0 |  |  |  | # 👈 
+| name | string |  | false | 10 | 100.0 |  |  |  |

As one solution, I've implemented this feature, but holding the types in a comma-separated format might be something we need to discuss. I would like to hear your opinion.

@k1LoW k1LoW added enhancement New feature or request minor labels Nov 23, 2023
@k1LoW
Copy link
Owner

k1LoW commented Nov 23, 2023

@mrtc0 Thank you for your GREAT proposal.

At this time, Type is not a multi-valued structure, so I think the comma delimitation is appropriate.

@k1LoW
Copy link
Owner

k1LoW commented Nov 23, 2023

Thank you!

@k1LoW k1LoW merged commit 0865747 into k1LoW:main Nov 23, 2023
2 checks passed
@github-actions github-actions bot mentioned this pull request Nov 23, 2023
@mrtc0 mrtc0 deleted the support-multitype-field-on-mongodb branch November 23, 2023 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request minor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants